Security of things, places, and people has long been a major area of patenting of methods, systems, techniques, and technology. The first mechanical locks for security of things were made of wood, and records show them in use some 4,000 years ago in Egypt; the first all-metal locks appeared about the year 890 in England. In the 18th century, the idea of a key-operated lock using circular patterned sliders was invented by the oldest lock company in the world, Bramah Security Equipment, which still makes these locks today. This idea developed to a circular disk type of padlock patented by R. K. Winning, in a U.S. Pat. No. 1,655,002, dated Jan. 3, 1928, which he followed with U.S. Pat. No. 1,736,183, dated Nov. 19, 1929, that was assigned to the Dudley Lock Corporation. This is the typical “high school locker, lock” used to this day by most school age children. Bunn, in 2001 U.S. Pat. No. 6,240,365 advances the security of things and people to drivers and their automobiles incorporating a plurality of sensors and devices including wireless communications, GPS location, and driver access key pad and credit card/smart card technologies.
The origin of securing things, places and people by people analyzing specific relevant information seems to have begun in Madrid, Spain, several centuries ago in which time the Spanish people did not trust mechanical locks. There, neighborhoods were guarded by patrolling watchmen carrying the keys to the buildings being patrolled. When anyone wished to enter a locked building, they had to loudly and noisily attract the attention of the watchman. The watchman would assess whoever was requesting entrance and, if approved, would permit access, relocking the building behind them. The assessment done by the watchman would likely include looking at the faces of the entrants to recognize them, listening to the reason for entrance and assessing the truth of the same, observing the nervousness of the entrants to detect deceit or lying, and likely viewing the surroundings for evidence of possible accomplices or potential threat.
Replacing patrolling watchman with security video cameras is well known. Lemelson, in 1991 U.S. Pat. No. 5,067,012, reveals a method and system for scanning and inspecting video camera images for automated recognition of objects and MacCormack, et al. in 2002 U.S. Pat. No. 6,031,573 reveal an intelligent video security camera management system for performing multiple comparative functions in parallel in order to reduce the volume of video tape recordings and reduce the time to retrieve information from those tapes. Hsieh, in 2002 U.S. Pat. No. 6,400,264, expands on the security camera applications by teaching an indoors surveillance camera in a building or the like in which intruders have entered and a “far end” remote, fixed or mobile, monitor that allows security personnel patrolling the neighborhood of the building, to observe conditions at the location of the camera from a safe location removed from the camera location. Hsieh teaches an intelligent camera that transmits its images to a “patrol box” which includes coded access for security personnel only so they can view the camera images stored at the patrol box and/or for them to control the camera.
Cotton, et al. in 1986 U.S. Pat. No. 4,630,110, teach a system and methods using a plurality of video surveillance cameras to monitor cash registers and cashiers for potential theft or coercive actions to fraudulently misrepresent billing for items being sold. This system uses intelligent controllers for displaying selected camera images either live or in playback of recorded images for viewing by security personnel. Everett, Jr., et al., in 1989 U.S. Pat. No. 4,857,912, teach an intelligent security assessment system which robotically patrols for fire, smoke, flooding and intrusion in a given area much like the Spanish watchman patrolling the neighborhood described earlier. This patrolling robot carries a multiplicity of intrusion sensors and integrating the input information from the sensors and weighing their sums relative to a reference level the system eliminates false detections; for a valid detection, it then automatically activates a security video camera on the robot and a monitor display remotely for security personnel observation. Chim, in 2001 U.S. Pat. No. 6,275,258, teaches a video camera which follows or “tracks” a speaking person as they move about, by pointing the camera in the direction of sound of the speech, in order to provide a means of keeping the image of a teleconferencing speaker in the visual center of the image being observed by a video camera.
The problem with all of these existing technologies is that they rely on detecting the physical presence of material conditions considered out of the ordinary, such as presence of smoke and heat indicating fire, or presence of people to indicate intrusion, and they usually rely on security personnel verification by viewing video camera data either live or in replay. This can be time consuming, and where automated continuous video display is used these technologies can lead to security personnel boredom resulting in overlooked or neglected observations and ineffective security.
Replacing the patrolling watchman, who listens to what a subject says in order to determine his intent or truthfulness, with automated voice stress analysis is also well known. Dudley, in his 1939 U.S. Pat. No. 2,181,265, described methods and apparatus for measuring and recording the voice of human subjects by microphone and amplifiers that were wired through a multiplicity of frequency channels to a multi-pen chart recording system for later observation by skilled interpreters. Dudley recognized that text of written or spoken words omit all those things that enter into the voice, such as stress, intonation, duration, brogues and accents, slurring and weakening of sounds and the various other characteristics which go to make up speech which he teaches can be interpreted from visual analysis of these frequency channel recordings. Fischer, in his 1965 U.S. Pat. No. 3,195,533, expanded upon the ideas of measuring the physiological and emotional nervous system stress and reaction of subjects by placing a plurality of bioelectric sensors on various locations on the skin of subjects and wiring these from the subjects to a chart recording system for later observation by skilled analysts.
Bell, Jr., et al., in 1976 U.S. Pat. No. 3,971,034, carry further the voice analysis for detecting psychological stress by displaying on chart recorders infrasonic voice modulations related to psycho-physiological state changes in subjects relating specifically to lie detection methods. Bogdashevsky, et al., in 1999 U.S. Pat. No. 6,006,188, advance the analysis function of voice stress analysis from the more manual chart reading analysis methods to a hardware, software and/or firmware system including a stored knowledge database of one or more speech models corresponding to a characteristic group of reference subjects to which the recording of a subject's speech can undergo comparative analysis for determining psychological or physiological stress characteristics. MacCaughelty, in 2000 U.S. Pat. No. 6,055,501, advances this voice stress analysis further to detect counter homeostasis oscillation perturbation signals within the wave form of human subjects that are said to reflect arousal or other biological processes in the autonomic nervous system.
The addition of textual information to that of voice stress to improve the detection of a potential security threat is also well known. Kucera, et al., in 1989 U.S. Pat. No. 4,868,750, teach a system in which a voice/text translator facility converts a subject's speech into text and then teaches a system of grammatical annotation of natural language with which each word of the subject's text is compared for analyses to constructed intelligent words and phrases to establish the meaning and intent of the subject's speech.
Walters, in 2002 U.S. Pat. No. 6,363,346, teaches combining the methods of text extraction from automated speech recognition analysis of a telephone calling subject's speech and the automated analysis of electronic voice signals of the subject's speech for voice stress analysis to predict the physiological state of the calling subject. Avrunin, et al., in 2002 U.S. Pat. No. 6,523,008, further advance these techniques to the application for truth-enabling internet communications of computer automated analyses again combining text extraction and voice stress analysis of the speech and text of internet subjects.
The above clearly indicates it is well documented and understood that measuring of physiological stress of people can permit the understanding of not only what a person (or persons) is saying but whether that person(s) is under stress when saying it, and that this can be related to the truthfulness of what is being said as well as related to the presence of a threat or a threatening intention from that person or persons.
The use of security surveillance video cameras to provide visual images to security personnel for monitoring people places and things is well known, as are the problems of boredom and neglect they cause the security personnel viewing the monitors displaying the views of such cameras. Automating the recognition of objects monitored by such video cameras is one step towards providing the security personnel with more intelligent information than just a repetitive, boring, mind numbing and mostly unchanging visual scene.